Too Good to Be Bad: the Effect of Favorable Expectations on Usability Perceptions

The most common measurements in the task-based usability evaluation paradigm include behavioral (e.g., completion times or errors) and subjective measures (e.g., ratings). Previous work has shown that success and performance in the test tasks do not dictate subjective usability perceptions, which instead are affected by appraisals of the system such as those of its aesthetic appeal. While these appraisals are an outcome of the exposure to and the interaction with the system, less is known about the effect of predispositions (expectations) formed before any exposure. To understand how expectations influence usability perceptions, we devised an experiment wherein 36 subjects read a positive or a negative product review for a novel mobile device (while a control group read nothing) before a usability test. The results demonstrate a surprisingly strong amplifying effect of the positive expectation on the post-experiment ratings, which, interestingly, held even in a condition where the users failed in all of the tasks. We briefly discuss implications of this finding.